Goto

Collaborating Authors

 individual contribution


AI-Driven Contribution Evaluation and Conflict Resolution: A Framework & Design for Group Workload Investigation

arXiv.org Artificial Intelligence

The equitable assessment of individual contribution in teams remains a persistent challenge, where conflict and disparity in workload can result in unfair performance evaluation, often requiring manual intervention - a costly and challenging process. We survey existing tool features and identify a gap in conflict resolution methods and AI integration. To address this, we propose a framework and implementation design for a novel AI-enhanced tool that assists in dispute investigation. The framework organises heterogeneous artefacts - submissions (code, text, media), communications (chat, email), coordination records (meeting logs, tasks), peer assessments, and contextual information - into three dimensions with nine benchmarks: Contribution, Interaction, and Role. Objective measures are normalised, aggregated per dimension, and paired with inequality measures (Gini index) to surface conflict markers. A Large Language Model (LLM) architecture performs validated and contextual analysis over these measures to generate interpretable and transparent advisory judgments. We argue for feasibility under current statutory and institutional policy, and outline practical analytics (sentimental, task fidelity, word/line count, etc.), bias safeguards, limitations, and practical challenges.


Crowd IQ -- Aggregating Opinions to Boost Performance

arXiv.org Artificial Intelligence

We show how the quality of decisions based on the aggregated opinions of the crowd can be conveniently studied using a sample of individual responses to a standard IQ questionnaire. We aggregated the responses to the IQ questionnaire using simple majority voting and a machine learning approach based on a probabilistic graphical model. The score for the aggregated questionnaire, Crowd IQ, serves as a quality measure of decisions based on aggregating opinions, which also allows quantifying individual and crowd performance on the same scale. We show that Crowd IQ grows quickly with the size of the crowd but saturates, and that for small homogeneous crowds the Crowd IQ significantly exceeds the IQ of even their most intelligent member. We investigate alternative ways of aggregating the responses and the impact of the aggregation method on the resulting Crowd IQ. We also discuss Contextual IQ, a method of quantifying the individual participant's contribution to the Crowd IQ based on the Shapley value from cooperative game theory.


Individual Contributions as Intrinsic Exploration Scaffolds for Multi-agent Reinforcement Learning

arXiv.org Artificial Intelligence

In multi-agent reinforcement learning (MARL), effective exploration is critical, especially in sparse reward environments. Although introducing global intrinsic rewards can foster exploration in such settings, it often complicates credit assignment among agents. To address this difficulty, we propose Individual Contributions as intrinsic Exploration Scaffolds (ICES), a novel approach to motivate exploration by assessing each agent's contribution from a global view. In particular, ICES constructs exploration scaffolds with Bayesian surprise, leveraging global transition information during centralized training. These scaffolds, used only in training, help to guide individual agents towards actions that significantly impact the global latent state transitions. Additionally, ICES separates exploration policies from exploitation policies, enabling the former to utilize privileged global information during training. Extensive experiments on cooperative benchmark tasks with sparse rewards, including Google Research Football (GRF) and StarCraft Multi-agent Challenge (SMAC), demonstrate that ICES exhibits superior exploration capabilities compared with baselines. The code is publicly available at https://github.com/LXXXXR/ICES.


Select to Perfect: Imitating desired behavior from large multi-agent data

arXiv.org Artificial Intelligence

AI agents are commonly trained with large datasets of demonstrations of human behavior. However, not all behaviors are equally safe or desirable. Desired characteristics for an AI agent can be expressed by assigning desirability scores, which we assume are not assigned to individual behaviors but to collective trajectories. For example, in a dataset of vehicle interactions, these scores might relate to the number of incidents that occurred. We first assess the effect of each individual agent's behavior on the collective desirability score, e.g., assessing how likely an agent is to cause incidents. This allows us to selectively imitate agents with a positive effect, e.g., only imitating agents that are unlikely to cause incidents. To enable this, we propose the concept of an agent's Exchange Value, which quantifies an individual agent's contribution to the collective desirability score. The Exchange Value is the expected change in desirability score when substituting the agent for a randomly selected agent. We propose additional methods for estimating Exchange Values from real-world datasets, enabling us to learn desired imitation policies that outperform relevant baselines. Imitating human behaviors from large datasets is a promising technique for achieving human-AI and AI-AI interactions in complex environments (Carroll et al., 2019;, FAIR; He et al., 2023; Shih et al., 2022). However, such large datasets can contain undesirable human behaviors, making direct imitation problematic. Rather than imitating all behaviors, it may be preferable to ensure that AI agents imitate behaviors that align with predefined desirable characteristics. In this work, we assume that desirable characteristics are quantified as desirability scores given for each trajectory in the dataset.


Measuring the Driving Forces of Predictive Performance: Application to Credit Scoring

arXiv.org Artificial Intelligence

In credit scoring, machine learning models are known to outperform standard parametric models. As they condition access to credit, banking supervisors and internal model validation teams need to monitor their predictive performance and to identify the features with the highest impact on performance. To facilitate this, we introduce the XPER methodology to decompose a performance metric (e.g., AUC, $R^2$) into specific contributions associated with the various features of a classification or regression model. XPER is theoretically grounded on Shapley values and is both model-agnostic and performance metric-agnostic. Furthermore, it can be implemented either at the model level or at the individual level. Using a novel dataset of car loans, we decompose the AUC of a machine-learning model trained to forecast the default probability of loan applicants. We show that a small number of features can explain a surprisingly large part of the model performance. Furthermore, we find that the features that contribute the most to the predictive performance of the model may not be the ones that contribute the most to individual forecasts (SHAP). We also show how XPER can be used to deal with heterogeneity issues and significantly boost out-of-sample performance.


Defining the skills citizens will need in the future world of work

#artificialintelligence

We know that digital and AI technologies are transforming the world of work and that today's workforce will need to learn new skills and learn to continually adapt as new occupations emerge. We also know that the COVID-19 crisis has accelerated this transformation. We are less clear, however, about the specific skills tomorrow's workers will require. Research by the McKinsey Global Institute has looked at the kind of jobs that will be lost, as well as those that will be created, as automation, AI, and robotics take hold. And it has inferred the type of high-level skills that will become increasingly important as a result. 1 1.


CSCR:Computer Supported Collaborative Research

arXiv.org Artificial Intelligence

It is suggested that a new area of CSCR (Computer Supported Collaborative Research) is distinguished from CSCW and CSCL and that the demarcation between the three areas could do with greater clarification and prescription. Keywords: HCI, CSCW, CSCL, CSCR 1. Introduction The twin fields of Computer Supported Collaborative Work (CSCW) and Computer supported Collaborative Learning (CSCL) have been the subject of intense interest in the HCI research community during the past seven years. The split between CSCW and CSCL has grown wider in response to the recognition that the learning process is more distinct from the working pattern and is more intensively understood through new theories of pedagogy and education. It has become apparent that CSCL requires all of the facets of CSCW but in addition is constraint by these pedagogical theories and as such it is argued here that CSCL is a subset of CSCW (see figure1) The process of research is also a learning process but one which is more highly refined and involves learning in a particular way using special techniques and tools. As such it is argued further that research which is supported by computer collaboration is a subset of CSCL (fig.1) Figure 1 2. Differences between CSCW and CSCL Diaper (2005) maintains that the History of HCI shows a lack of coherent development.